Bayesian Sequential Decision Processes with Censored Observations: Derivative Analysis and Applications

Censored (or truncated) observations are quite prevalent in practice. How does the existence of imperfect observations affect optimal decisions? We consider this problem in the setting of a general finite-horizon Bayesian sequential decision process. We first prove a general derivative result that resembles the classic envelope theorem. We then show that for a class of Bayesian sequential decision processes, there exists a recursive formula for the first-order derivative of the Bayesian dynamic programming objective function. With this formula, the derivative can be computed by a simple backward iteration, and the optimal decision in each period can be determined based on the first-order condition. For illustrative purposes, we give three application examples in the contexts of dynamic pricing, online auction, and optimal inventory control. The insights obtained from these applications all suggest that the existence of imperfect observations tends to shift optimal decisions in the direction of reducing the effect of censoring.